Home

Introduction

Xposome is a shiny application that aims to build a repository of gene expression datasets that profile the transcriptomic response to exposure of known toxins.

There are two types of Xposome applications that you can install on your local machine:

  • Xposome with GeneHive database incorporated
  • Xposome without GeneHive

In this post, I will only go over instructions of how to install Xposome without GeneHive on your local machine. If you are interested in incorporating GeneHive as a database, please contact us for further implementation.

Local Installation

Must have R and Rstudio installed before proceeding:

To run Xposome application locally:

  • Download the source code from GitHub repo
  • Unzip the file and navigate to shinyApps folder
  • Open app.R within that folder
  • Click Run App button to launch the application in Rstudio

Dockerize Xposome Application

Not familiar with Docker? You can download it here and check out the R Docker tutorials.

To dockerize the Xposome application:

  • Download the source code from GitHub repo
  • Unzip the file and open the terminal. Use the cd command to navigate to where the dockerfile is stored. The dockerfile is a text file that contains all of the dependencies and instructions on how to run the Xposome app.
  • Run the following command to build the docker image for the Xposome application
    docker build xposome .
    To check if the image is built succesfully,
    docker images
  • Create a docker container that contains the Xposome image and publish the application on the host machine.
    docker run -d -p 3838:3838 \
      -v shinyApps/:/srv/shiny-server/ \
      -v shinyApps-log/:/var/log/shiny-server/ \
      xposome:latest
    -d is to run the container in detached mode
    -v is to mount the code base or data files from the host directory onto the container directory <host_dir>:<container_dir>
    -p is to link the container port onto the host port and publish the application on the host domain <host_port>:<container_port>
    For more information about the syntax usage, see docker documentation

    To check if the container is built and running sucessfully,
    docker container ps
  • Once the app is up and running, you can visit the local host on your machine, i.e. http://localhost:3838 to vertify if the application is indeed hosted there.

The docker image for Xposome is also available at Docker Hub. To pull the image, just type the following command in the terminal:

docker pull montilab/xposome

Additionally, check out how you can configure your web applications with HTTPS traffics with NGINX and Docker-Compose

Data Access

The Xposome application requires several structural datasets in order for it to run smoothly. We provide API access to our data, see API Explorer.

Application Usage

There are two main parts of the Xposome application:

  • Project Page
  • Moderator Page

The Project Page

Contains a list of projects or chemical screenings that were exposed to high-throughput transcriptomics assays. Each screen provides annotation about the chemicals and cell-line samples that were used in the experiment. A detailed analysis was also done to explore gene expression set, gene set enrichment, and gene connectivity that was exposed to a panel of known carcinogens.

Below is a screenshot of our home page and a snapshot of the Adipogenicity project:

The Moderator Page

Requires access from authorized users. This restricted page only allows certain users to make changes to the existed screens or to upload new screens onto the Project Page.

The default login credentials are set as:

  Username: Xposome
  Password: Xposome

Lastly, check out the documentation of how to reset password if users forgot their login credentials using Send Emails with Postfix.



Links

License

Developers

NGINX and Docker-Compose

NGINX & Docker-Compose

NGINX - An open source web server and reverse proxy technology used for hosting websites and applications

Docker-Compose - A technology for enabling docker containers to communicate to each other

In this post, I will go over steps of how to create an encrypt (https) traffic that allows users to interact with the Xposome application that is published through a docker container. To achieve that, we need both NGINX and docker-compose.

Here are seven steps to set-up NGINX and docker-compose:

  1. Install docker-compose
  2. Build a NGINX image from Docker Hub without the need of installing NGINX software
  3. Dockerize the Xposome application locally or pull the image from Docker Hub
  4. Set up NGINX configuration files
  5. Create a YAML file for docker-compose (i.e. docker-compose.yml)
  6. Run the docker-compose.yml file
  7. Check to see if the application is running on https


Step 1: Install docker-compose

Check out how to install Docker-compose here

To check if docker-compose is installed, 

docker-compose --version

Step 2: Build a docker image for NGINX

An image for NGINX can be found at Docker Hub. You can pull this image without the need of installing the NGINX software locally.

docker pull nginx:latest

To check if the image is built successfully 

docker images

Also see this link on how to use the NGINX image

Step 3: Dockerize the Xposome application locally or pull the image from Docker Hub

You can check out this step on my previous post here for more details

Step 4: Set-up NGINX configuration files

Once the images for NGINX and Xposome application are build successfully, we can start setting up the NGINX configuration files. There are two configuration files for NGINX, nginx.conf and default.conf.template files.

The nginx.conf file contains the standard configuration for NGINX. Unless you know how to add directives to a configuration file, otherwise, I do not recommend making any changes to this file.

For example, one of the directives that we used on our web portal is to maximize a file uploading size from our users. This directive is useful as it helped us avoided the problems of importing some gene expression datasets that far exceeded the limit of 1MB file uploads on NGINX.

The default.conf.template file contains HTTP configuration that allows NGINX to direct any encrypt (https) or unencrypt (http) traffics to an application that is published on a specified port on the host machine.

How NGINX talks to Docker

With NGINX,
nginx → listens on → port 80 (http) and 443 (https) 

With Docker,
Docker → listens on → port 80/443/8080/3838/8000/… 

With SHINY
Shiny → listens on localhost → port 3838/8787/4848/…

In other words, when we run a docker container, we basically expose our application through a port that is viewable on a Shiny's localhost (for example: port 3838). Docker listens to this port and reroutes the application to a specified port on the host machine (for example: port 7856). Therefore, when we navigate to port 7856 on the host domain (for example http://[domain_name/ip address]:7856), we will see that the shiny app is hosted there.

Nevertheless, we probably do not want to direct our users to an application through a port link. Thus, we can use NGINX configuration files to create an alias location for the application (for example https://[domain_name/ip address]/Xposome/). Hence, when a user is navigated to that specified address on the host domain, NGINX will transfer that traffic to the port e.g. 7856 of where the application was originally published on. Furthermore, NGINX will encrypt that traffic with a https protocol. As a result, NGINX is served as a reverse proxy for hosting our applications.

Here is a snapshot of how the http configuration file looks like:

Step 5: Create a docker-compose.yml file

Once we had configured NGINX as a reverse proxy for our application, we can create a docker-compose.yml file that comprises of all of the containers that we want to communicate with NGINX container.

To enable communication between containers, in the docker-compose file, we must create a service to run the NGINX container and a service to run the Xposome container. Under each service, we need to specify a list of instructions of how a container can be built, for instance, what image is used to build the container, what do we want to name the container, what port is used to expose the shiny app to Docker, and what port is used to publish the app on the host machine, etc.

After we had defined all of the key components of how the containers can be built, one last step is to confirm the host port on the docker-compose.yml file matches the port that NGINX redirects the encrypt traffics to the application on HTTP configuration file.

Here is a snapshot of the docker-compose.yml file:

Step 6: Run the docker-compose file

After we had the docker-compose file and NGINX configuration files all set up, we can cd to where docker-compose.yml is located and run the following command to fire up all the containers that were defined in the docker-compose file.

docker-compose up -d

-d is used to run docker compose in detach mode

Step 7: Check to make sure the application is running on https

Navigate to the alias domain for the application (for example https://montilab.bu.edu/Xposome/) and see if the application is indeed hosted there.



Links

License

Developers

Send Emails with Postfix

Send Emails with Postfix

Postfix - A popular open-source Mail Transfer Agent (MTA) that can be used to route and deliver emails on a Linux system

Docker - A software platform that allows developers to build, test, deploy, and run applications using containers

In this post, I will go over steps of how to send emails from a docker container through Postfix installed on the host machine. In my previous posts, I went over how to dockerize the Xposome application. The Xposome application has two parts: the Project Page and the Moderator Page. In the Moderator Page, we have an option that allow users to retrieve their passwords if they forgot their login credentials. Postfix is actually used here to send a notification email to users and remind them that their password has changed.

Here are five steps to set-up Postfix and send new password to users if they forgot their login credentials:

  1. Install Postfix
  2. Dockerize the Xposome application locally or pull the image from Docker Hub
  3. Configure Postfix to listen to requests from docker container
  4. Write an email to test the Postfix Mail Server
  5. Check to see if the email is sent correctly

Step 1: Install Postfix

Depending on your Operating System, for mine, I have CentOS 8 server. Therefore, I will use this documenation on how to install and configure postfix mail server on CentOS 8. You can also see this link to install and configure Postfix on Ubuntu 16.04.

Step 2: Dockerize the Xposome application locally or pull the image from Docker Hub

See my previous post here for more details

Step 3: Configure Postfix to listen to requests from docker container

After Postfix is installed, the main configuration files is stored at /etc/postfix/main.cf.

To configure Postfix to listen to requests from docker container, there is something called docker bridge (docker0) which acts as a bridge between the ethernet port and the docker container so that data can go back and forth. Hence, we want Postfix to listen on the docker0 interface. To do that, type ifconfig on your host system to find out the bridge address and set your Postfix to listen on it.

As you can see in the image above, the IP address is 172.17.0.1

In the Postfix configuration file (/etc/postfix/main.cf), set

inet_interfaces = 172.17.0.1

While you are there, add your actual docker container ip address to mynetworks

mynetworks = 172.17.0.5

172.17.0.5 is the private IP address of my docker container from which I use to send emails.

You can find the IP address for your docker container by

docker inspect [container_id/container_name]

Step 4: Write an email to test the Postfix Mail Server

Here is an email function that sends a temporary password to users who forgot their login credentials:

sendpassword <- function(from_sender="montilab@bu.edu", to_recipient="montilab@bu.edu", recipient_first="Montilab", recipient_last="Montilab", recipient_account="Montilab", tmp_pwd){

  recipient=paste(recipient_first, recipient_last)

  msg <- mime_part(
    paste0(
      '<!DOCTYPE>',
      '<html>',
      '<head>',
      '<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>',
      '<meta name="viewport" content="width=device-width, initial-scale=1.0"/>',
      '<title>HTML MESSAGE</title>',
      '<style type="text/css">',
      '</style>',
      '</head>',
      '<body>',
      '<p>Hi <strong>', recipient_first, ',</strong></p>',
      '<p>The password for your Xposome account has changed.</p>',
      '<p></p>',
      '<p>Username: <strong>', recipient_account, '</strong></p>',
      '<p>Temporary password: <strong>', tmp_pwd, '</strong></p>',
      '<br>',
      '<p>Log back in? Follow this link, <strong>https://montilab.bu.edu/Xposome/?page=sign_in</strong></p>',
      '<br>',
      '<p>Best,</p>',
      '<p>Montilab Team</p>',
      '</body>',
      '</html>' 
    )
  )

  ## Override content type.
  msg[["headers"]][["Content-Type"]] <- "text/html"

  from <- paste0("\"Montilab Team\"<", from_sender, ">")
  to <- paste0("\"", recipient, "\"<", to_recipient, ">", collapse="")
  subject <- "Temporary password for Xposome"
  body <- list(msg)
  sendmail(from, to, subject, body, control=list(smtpServer="smtp.bu.edu", smtpPort="25"))

}

On the Xposome application, there is a sign in page. When a user clicks on the forgot password? link, a modal dialog will pop up. After he/she provided the user account information and click the submit button, a new temporary password will be send to his/her email.

Here a snapshoot of the Xposome interface:

Step 5: Check to see if the email sent correctly

if you are using the same email format that I have structured above, you should receive an email similar to this one.



Links

License

Developers

API Calls

API EXPLORER

API EXPLORER

The API Explorer provides documentation on how users can retrieve data from the Xposome application and create their next project. All successful responses are returned in JSON. Only queries that respond with a 200 response code is considered successful.

Here are standard HTTP codes you will find in the “Status” element of the response body.

Status Code Description
200 OK Standard HTTP successful response
404 Bad Request or Source Not Found Standard HTTP invalid request response
500 Internal Server Error There was an unexpected error on our server. If you see this error, please help us notify the issue on our GitHub page


There are four data sources that are currently returned from the API.

Data Description Return
Projects a list of projects in the GeneHive Database. Visit our webpage to see on how we define each project list
Datasets a list of data sources for generating the Xposome portal such as profile and chemical annotation, expression set, gene set enrichment, connectivity set, and K2-taxonomer data frame or RDS file
RDS Bundle a bundle of RDS files for a given Xposome projects, including profile annotation, chemical annotation, expression set, gene set enrichment, connectivity set, and K2-taxonomer zip file
Statistics a summary statistics for a specific chemical within a particular project. The statistics are calculated based on its gene expression, gene set enrichment and gene connectivity data frame


Below are a list of R packages and GET requests in R to retrieve data from the Xposome projects. However, one can use Python or other programs to implement the REST API calls.

# R packages required for API calls
library(K2Taxonomer)
library(Biobase)
library(jsonlite)
library(httr)

Projects

/projects Return a list of projects available in the GeneHive database


Datasets

/profile_annotation Return the profile annotation of a specific project


/chemical_annotation Return the chemical annotation of of a specific project


/expression_set Return the gene expression dataset of a specific project


/enrichment_set Return the enrichment dataset of a specific project


/connectivity_set Return the connectivity dataset of a specific project


/k2_taxonomer Return the K2Taxonomer dataset of a specific project


RDS Bundle

/rds_bundle Return a bundle of RDS files for a given project


Statistics

/gene_expression Return a collection of differential expressed genes that exposed to a known toxin


/gs_enrichment Return a collection of gene set enrichment that was exposed to a given drug


/connectivity Return a collection of gene connectivity that are linked to exposure of a known chemical


Additionally, we provide direct links to query contents on our website. See Links to Application.



Links

License

Developers